Fixed and Random Effects Models

How should we pool effect sizes in meta-analysis?

Jason Moggridge

2024-07-30

Meta-analysis Recap

Meta-analysis Research

  • To estimate the true effect size from multiple studies:

    • define research question (PICOT)
    • collect results from appropriate studies
    • apply statistical model*

Meta-analysis Data

  • unit of observation is the study; dataset is \(K\) studies
  • variables:
    • estimated effect size (\(\widehat{\theta}_{k}\))
    • standard error (\(\epsilon_{k}\))
    • sample size (\(N_{k}\))

How should we pool effect sizes?

  • How do we estimate what the true effect is?
  • How should we weigh estimates?

Meta-analysis Models

  • Fixed effects and random effects models make different assumptions about how data are generated.
  • The chosen model should be appropriate for the underlying processes

Fixed Effects Model

Fixed Effects Assumption

  • One true effect size (\(\theta_{\ F }\))
  • Deviation from true effect is due to sampling error only

  • One true effect size that exists for the population
  • Errors are just due to random sampling (noise).

Fixed Effects Model


  • Sampling error \(\epsilon_{\ k }\) is represented by the std. error \(SE_{k}\)
  • SE is inversely related to sample size \(N_{k}\).
  • \(\widehat{\theta}_{k}\) with small SE should be closer to true \(\theta\)
  • These \(\widehat{\theta}_{k}\) should be given more weight

Inverse-Variance Weighting


Study weight \(w_{k}\) is inversely related to \(SE^2\)

\[ w_{k}=\frac{1}{SE^{2}_{k}} \]

The pooled estimate \(\widehat{\theta}\) is the weighted average:

\[ \widehat{\theta} = \frac{\Sigma_{k=1}^{K} \widehat{\theta}_{k} w_{k}}{\Sigma_{k=1}^{K} w_{k}} \]

Fixed Effects Model Interpretation

  • Assumes all studies are sampling a completely homogenous population and measurement is performed in exactly the same manner.
  • Model doesn’t allow between-study heterogeneity
  • Too simplistic to account for real differences between studies in the underlying true effect.

Random Effects Model

Random Effects Assumption

  • There are are universe of possible studies,
  • Each study is different with it’s own true effect size
  • Studies are randomly drawn from a distribution with mean true effect size \(\mu\)

Random Effects Model

A study’s effect size is an estimate of the true study effect size plus sampling error \[ \widehat{\theta}_{k}=\theta_{k} +\epsilon_{k} \]

The true effect size of study k is drawn from a distribution of effect sizes with mean \(\mu\), with error \(\zeta_{k}\).

\[ {\theta}_{k}=\mu+\zeta_{k} \]

Random Effects Model

  • REM as a hierarchical structure

REM weighting

  • we must first establish the variance of the distribution of true effect sizes, \(\tau^2\)
  • this value informs the between-study heterogeneity for weighting studies

\[ w^{*}_{k}=\frac{1}{s^{2}_{k} + \tau^2} \]

And then we estimate the overall mean effect size using the same weighted average as before.

Estimating \(\tau^2\)

  • Many procedures exist, all of which are far too complicated for this talk!
  • The DerSimonian and Laird method is one of the oldest and very common

Exchangeability Assumption

  • \(\zeta_{k}\) is independent of \(k\)
  • there is no way to know what \(\zeta_{k}\) is a priori, ie. without looking at the data
  • we say that different studies are assumed to be ‘exchangeable’ because we have no way to know \(\zeta_{k}\)
  • This assumption may not be true…

Should we just always use the REM?


  • Only choose the FEM if you have clear reason to (rare)

  • For most scenarios, use the REM.

A Test for Between-Study Heterogeneity


A simple way to determine whether the REM is appropriate is to test whether the confidenece interval for \(\tau^2\) includes zero or not.

References & Further Reading